JOURNAL OF LEARNING ANALYTICS 

(2014). Characterizing User Behaviours and Products in an Online Educational Community: A Comparison between Novices and Elders. Journal 
of Learning Analytics, 1(3), 150-153. 


S • LAR 


Characterizing User Behaviours and Products in an 
Online Educational Community: A Comparison 
between Novices and Elders 


Min Yuan and Mimi Recker 

Utah State University 
mimi.recker@usu.edu 

ABSTRACT: This study focuses on two user types in an online educational community — novices 
and elders — and explores the characteristics of their behaviours and contributed products by 
analyzing usage logs. Results show that novices and elders have different behavioural 
trajectories. In addition, the products created by elders attract more views. 
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1 INTRODUCTION 

Many online educational communities (OECs), such as the Instructional Architect and Tapped In, have 
been established to facilitate user knowledge building, promote mutual interaction, and meet 
educational needs (Preece & Maloney-Krichmar, 2003; Recker, 2006; Suthers & Dwyer, 2014). By 
examining time spent in an OEC and the levels of participation, different user types can be identified. For 
example, we define novices as new members of an OEC, and elders as users who have participated in an 
OEC for some time and make frequent contributions (Bishop, 2007). Investigating the differences 
between novices and elders is important, as it allows researchers to understand the evolution of user 
behaviours and characteristics of the products they contribute (e.g., discussions, comments, educational 
artifacts). It also offers implications on the sustainability of OECs. 

The purpose of this study is to identify the differences between novices and elders based on the usage 
logs from a free, web-based tool called the Instructional Architect (IA.usu.edu). The IA allows teachers to 
build shared repositories of educational content and support each other. Teachers can contribute to 
repositories by creating new products, modifying existing products, and adding external resources to 
their products (Recker, 2006). They can support each other by publishing their products, so other users 
can view and even copy their products. In particular, this study compares the behaviours and resulting 
products between 386 novices who created accounts in the IA during 2013 and 152 elders who created 
accounts before 2013 and actively participated during 2013. 

2 THEORETICAL CONTEXT 

The rapid growth in Internet technologies has given rise to many OECs, which provide features to meet 
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teachers' instructional needs (Andrews, Preece, & Turoff, 2001). The way users use features of an OEC 
varies. For example, some prefer to create products using their own knowledge, some tend to build on 
and remix external resources in their products, while others may simply view other users' products 
(Yuan & Recker, 2014). In addition, user behaviours in an OEC are dynamic across time. For example, 
novices may be initially attracted to an OEC, and some eventually become key contributors and elders, 
while others leave after a short time (Iriberri & Leroy, 2009). In this way, the dynamics of user 
behaviours can affect the sustainability of the whole community. 

Further, Farzan, DiMicco, & Brownholtz (2009) noted that users' products can affect the sustainability of 
an OEC, as these products can help attract new users and influence existing users to remain in the 
community. Thus, describing user behaviours and characterizing users' products can lead to a better 
understanding of the sustainability of an OEC. Flowever, little research has focused on both aspects, 
especially drawing upon the usage logs of a specific community. In response, this study compared 
behaviours and products between novices and elders, and concludes with suggestions on how to 
improve the sustainability of OECs. 

3 METHOD 

This study addresses two questions oriented towards the IA community: 1) Flow did novices differ from 
elders in terms of their behaviours in 2013? 2) Flow did novices differ from elders in terms of the 
products each group created during 2013? We collected data from the IA database and analyzed it using 
descriptive statistics. We also used the Mann-Whitney test due to non-normal distribution. 

4 RESULTS 

4.1 A Comparison of User Behaviours 

First, on average, novices created more products, but made fewer modifications and incorporated fewer 
external resources than elders (see Table 1). Flowever, the Mann-Whitney test reveals that these 
differences are not significant. Second, novices and elders show different behavioural trajectories. In 
general, novices exhibited high activities after creating their accounts, after which their activities 
decreased. For example, novices created almost 50% of their products during the month they joined the 
community. On the contrary, elders made steady contributions to the IA throughout 2013. 


Table 1: Comparison of behaviours between novices and elders. 


Metrics 

Novices (N=386) 

Elders (N=152) 

Mean 

Median 

SD 

Mean 

Median 

SD 

# of products created 

1.32 

0 

2.50 

1.16 

0 

2.43 

# of modifications of products 

22.38 

0 

45.37 

23.51 

0 

53.75 

# of external resources added to products 

3.62 

0 

11.08 

5.65 

0 

14.60 
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4.2 A Comparison of Resulting Products 

During 2013, novices created 510 products and elders created 177 products, but elders' products were 
viewed significantly more times (U = 31637.50, p <.01; see Table 2). This suggests that elders 
contributed products that are more beneficial to users. 


Table 2: Comparison of products between novices and elders. 


Metrics 

Novices' products 

N=510) 

Elders' products (N=177) 

Mean 

Median 

SD 

Mean 

Median 

SD 

# of times products being viewed* 

12.41 

3 

43.25 

133.32 

17 

388.04 

# of times products being copied 

.01 

0 

.11 

.03 

0 

.20 


* Difference between the two groups is significant (Mann-Whitney test; p < .01) 

5 CONCLUSION 


This study offers a series of suggestions on improving the sustainability of OECs, such as providing 
support and dissemination to retain novice interest, especially within the first few months after they join 
the community, and highlighting elders' products to attract more new users. Furthermore, this study 
contributes to learning analytics, as it offers examples of how to trace and compare user behaviour and 
users' products based on usage logs (Ferguson & Buckingham Shum, 2012). This work is part of an 
ongoing series of studies; a forthcoming study uses logistic regression to investigate which variable(s) 
predict whether novices remain in the IA community. 
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